{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 202 Variable\n",
    "\n",
    "View more, visit my tutorial page: https://morvanzhou.github.io/tutorials/\n",
    "My Youtube Channel: https://www.youtube.com/user/MorvanZhou\n",
    "\n",
    "Dependencies:\n",
    "* torch: 1.0.0\n",
    "\n",
    "Variable in torch is to build a computational graph,\n",
    "but this graph is dynamic compared with a static graph in Tensorflow or Theano.\n",
    "So torch does not have placeholder, torch can just pass variable to the computational graph.\n",
    "\n",
    "https://ptorch.com/docs/4/pytorch-video-variable\n",
    "\n",
    "神经网络中的参数都是 Variable 变量的形式。Pytorch 0.4以后，Variable已经和Tensor进行了整合，可以直接用tensor，tensor的用法和variable相同。  \n",
    "\n",
    "要想使得Tensor具有autograd功能，只需要设置`tensor.requries_grad=True`。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "~~`autograd.Variable`是Autograd中的核心类，它简单封装了Tensor，并支持几乎所有Tensor的操作。Tensor在被封装为Variable之后，可以调用它的`.backward`自动计算所有的梯度，来实现反向传播，自动计算所有梯度~~ ~~Variable的数据结构如图所示。~~\n",
    "\n",
    "~~Variable主要包含三个属性。~~\n",
    "~~- `data`：保存Variable变量中的原始张量Tensor~~\n",
    "~~- `grad`：保存`data`对应的梯度，`grad`也是个Variable，而不是Tensor，它和`data`的形状一样。~~\n",
    "~~- `grad_fn`：指向一个`Function`对象，这个`Function`用来反向传播计算输入的梯度，具体细节会在下一章讲解。~~\n",
    "\n",
    "![图2-6:Variable的数据结构](images/autograd_Variable.png)\n",
    "\n",
    "  *从0.4起, Variable 正式合并入Tensor, Variable 本来实现的自动微分功能，Tensor就能支持。还是可以使用Variable(tensor), 但是这个操作其实什么都没做。建议以后直接使用 tensor*. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[1., 2.],\n",
       "        [3., 4.]], requires_grad=True)"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import torch\n",
    "\n",
    "# tensor = torch.tensor([[1,2],[3,4]]) # int\n",
    "tensor = torch.Tensor([[1,2],[3,4]]) # float\n",
    "\n",
    "# only Tensors of floating point dtype can require gradients\n",
    "tensor.requires_grad = True\n",
    "tensor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "计算梯度值用：`v_out.backward()`  \n",
    "查看梯度值用：`variable.grad`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(7.5000, grad_fn=<MeanBackward1>)\n",
      "tensor([[1.5000, 3.0000],\n",
      "        [4.5000, 6.0000]])\n"
     ]
    }
   ],
   "source": [
    "s_out = torch.mean(tensor*tensor)\n",
    "print(s_out)\n",
    "s_out.backward()\n",
    "print(tensor.grad)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[1., 2.],\n",
      "        [3., 4.]], requires_grad=True)\n",
      "tensor([[1., 2.],\n",
      "        [3., 4.]])\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([[1., 2.],\n",
       "       [3., 4.]], dtype=float32)"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print(tensor) # torch.Tensor\n",
    "print(tensor.detach()) # torch.Tensor\n",
    "tensor.detach().numpy() # numpy.ndarray"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 如下为Pytorch 0.4版本"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "from torch.autograd import Variable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[1., 2.],\n",
      "        [3., 4.]])\n",
      "tensor([[1., 2.],\n",
      "        [3., 4.]], requires_grad=True)\n"
     ]
    }
   ],
   "source": [
    "# torch.Tensor([[1,2],[3,4]]) # int\n",
    "tensor = torch.FloatTensor([[1,2],[3,4]])\n",
    "variable = Variable(tensor, requires_grad=True)\n",
    "\n",
    "print(tensor)\n",
    "print(variable)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Till now the tensor and variable seem the same.\n",
    "\n",
    "However, the variable is a part of the graph, it's a part of the auto-gradient.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(7.5000)\n",
      "tensor(7.5000, grad_fn=<MeanBackward1>)\n"
     ]
    }
   ],
   "source": [
    "t_out = torch.mean(tensor*tensor)       # x^2\n",
    "v_out = torch.mean(variable*variable)   # x^2\n",
    "print(t_out)\n",
    "print(v_out)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "v_out.backward()    # backpropagation from v_out"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$ v_{out} = {{1} \\over {4}} sum(variable^2) $$\n",
    "\n",
    "the gradients w.r.t the variable, \n",
    "\n",
    "$$ {d(v_{out}) \\over d(variable)} = {{1} \\over {4}} 2 variable = {variable \\over 2}$$\n",
    "\n",
    "得到每个variable对应的梯度值，let's check the result pytorch calculated for us below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0.5000, 1.0000],\n",
       "        [1.5000, 2.0000]])"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "variable.grad"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[1., 2.],\n",
      "        [3., 4.]], requires_grad=True)\n",
      "tensor([[1., 2.],\n",
      "        [3., 4.]])\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([[1., 2.],\n",
       "       [3., 4.]], dtype=float32)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print(variable) # torch.Tensor\n",
    "print(variable.detach()) # torch.Tensor\n",
    "variable.detach().numpy() # numpy.ndarray"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that we did `.backward()` on `v_out` but `variable` has been assigned new values on it's `grad`.\n",
    "\n",
    "As this line \n",
    "```python\n",
    "v_out = torch.mean(variable*variable)\n",
    "``` \n",
    "will make a new variable `v_out` and connect it with `variable` in computation graph."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Tensor"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(v_out)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Tensor"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(v_out.data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
